Project Overview/Objectives:
Context:
There is a huge demand for used cars in the Indian Market today. As sales of new cars have slowed down in the recent past, the pre-owned car market has continued to grow over the past years and is larger than the new car market now. Cars4U is a budding tech start-up that aims to find footholes in this market.
In 2018-19, while new car sales were recorded at 3.6 million units, around 4 million second-hand cars were bought and sold. There is a slowdown in new car sales and that could mean that the demand is shifting towards the pre-owned market. In fact, some car sellers replace their old cars with pre-owned cars instead of buying new ones. Unlike new cars, where price and supply are fairly deterministic and managed by OEMs (Original Equipment Manufacturer / except for dealership level discounts which come into play only in the last stage of the customer journey), used cars are very different beasts with huge uncertainty in both pricing and supply. Keeping this in mind, the pricing scheme of these used cars becomes important in order to grow in the market.
As a senior data scientist at Cars4U, you have to come up with a pricing model that can effectively predict the price of used cars and can help the business in devising profitable strategies using differential pricing. For example, if the business knows the market price, it will never sell anything below it.
Objective:
To explore and visualize the dataset, build a linear regression model to predict the prices of used cars, and generate a set of insights and recommendations that will help the business.
Data Dictionary (Feature : Explaination )
The data contains the different attributes of used cars sold in different locations. The detailed data dictionary is given below.
S.No. : Serial number
Name : Name of the car which includes brand name and model name
Location : Location in which the car is being sold or is available for purchase (cities)
Year : Manufacturing year of the car
Kilometers_driven : The total kilometers (a unit used to measure length or distance) driven in the car by the previous owner(s)
Fuel_Type : The type of fuel used by the car (Petrol, Diesel, Electric, CNG, LPG)
Transmission : The type of transmission used by the car (Automatic/Manual)
Owner : Type of ownership
Mileage : The standard mileage offered by the car company in kmpl or km/kg
Engine : The displacement volume of the engine in CC
Power : The maximum power of the engine in bhp
Seats : The number of seats in the car
New_Price : The price of a new car of the same model in INR Lakhs (1 Lakh INR = 100,000 INR)
Price : The price of the used car in INR Lakhs
Scoring Rubric & Best Practices for Notebook
The notebook should be well-documented, with inline comments explaining the functionality of code and markdown cells containing comments on the observations and insights.
The notebook should be run from start to finish in a sequential manner before submission.
It is preferable to remove all warnings and errors before submission. The notebook should be submitted as an HTML file (.html) and NOT as a notebook file (.ipynb).
- Define the problem and perform an Exploratory Data Analysis: Problem definition, questions to be answered - Data background and contents - Univariate analysis - Bivariate analysis (6 points)
- Illustrate the insights based on EDA: Key meaningful observations on the relationship between variables (3 points)
- Data pre-processing: Prepare the data for analysis and modeling - Missing value Treatment - Outlier Treatment - Feature Engineering (10 points)
- Model building - Linear Regression: Build the model and comment on the model performance (8 points)
- Model performance evaluation: Evaluate the model on different performance metrics (4 points)
- Actionable Insights & Recommendations: Conclude with the key takeaways for the business (5 points)
- Notebook - Overall Quality: Structure and flow - Well-commented code (4 points)
In [1]:
# importing all of the required libraries
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
%matplotlib inline
import seaborn as sns
from sklearn.linear_model import LinearRegression
from sklearn.model_selection import train_test_split
from sklearn.metrics import mean_squared_error
In [2]:
# loading the data set
pd.set_option('display.max_columns', None)
from google.colab import files
data_to_load = files.upload()
In [3]:
import io
df = pd.read_csv(io.BytesIO(data_to_load['used_cars_data.csv']))
Objective:
- Analyze and get an understanding of the dataset
- Prepare the dataset for modeling
- Create a model to predict the price of a used car, evaluate the model based on different metrics, and conclude with key business insights
We will start with EDA and Data Preprocessing to understand the data and prepare it for modelling.
Taking an initial look at the dataset¶
In [4]:
# checking the shape of the data set
print(f'There are {df.shape[0]} rows and {df.shape[1]} columns.')
In [5]:
df.info()
In [6]:
df.dtypes
Out[6]:
In [7]:
df.isnull().sum().sort_values(ascending=False)
Out[7]:
In [8]:
# checking for duplicate entries in the data set
df.duplicated().sum()
Out[8]:
In [9]:
# dropping duplicates
df.drop_duplicates(inplace=True)
Observations
- There are 7253 rows and 14 columns in the data set.
- There are 3 columns of int datatype (S.No., Year, Kilometers_Driven).
- There are 8 columns of object datatype (Name, Location, Fuel_Type, Transmission, Owner_Type, Mileage, Engine, Power).
- There are 3 columns of float datatype (Seats, New_Price, Price).
- There are 5 columns in the dataset with missing values (Price; 1234, Power: 175, Seats: 53, Engine: 46, Mileage: 2).
In [10]:
np.random.seed(1)
df.sample(n=10)
Out[10]:
In [11]:
df.describe(include='all').T
Out[11]:
Observations
- 2041 unque values for Name, most frequent is Mahindra XUV500 W8 2WD.
- 11 unique values for Location, most frequent is Mumbai.
- Values for Year range from 1996-2019.
- Values for Kilometers_Driven range from 171-6,500,000.
- 5 unique values for Fuel-Type, most frequent is Diesel.
- 2 unique values for Transmission, most frequent is Manual.
- 4 unique values for Owner_Type, most frequent is First.
- 450 unique values for Mileage, most frequent is 17.0 kmpl.
- 150 unique values for Engine, most frequent is 1197 CC.
- 385 unique values for Power, most frequent is 74 bhp.
- Values for Seats range from 0-10.
- Values for New_Price range from 3.91-375.0 INR Lakhs.
- Values for Price range from 0.44-160.0 INR Lakhs.
Now we will modify the individual columns as neccessary to prepare it for visualization and modeling.¶
In [12]:
# Dropping serial number column because it will not be neccessary for the model
df.drop(['S.No.'], axis=1, inplace=True)
In [13]:
# Creating a new column for car age
current_year = 2022
df['Car_Age'] = current_year - df['Year']
df.head()
Out[13]:
In [14]:
# Converting year column to datetime datatype
df['Year'] = pd.to_datetime(df['Year'], format='%Y')
In [15]:
#checking our changes
df.head()
Out[15]:
In [16]:
# Splitting the name coluumn into Make and Model columns
# creating a column for car model
df['Car_Model'] = df.Name.str.split().str[0:3]
# joining the split list values in the car model column
df['Car_Model'] = df.Car_Model.apply(lambda x: ' '.join(x))
# creating a column for car brand
df['Car_Brand'] = df.Name.str.split().str[0]
# dropping the initial Name column
df.drop(['Name'], axis=1, inplace=True)
# taking a look at the changes made
df.head()
Out[16]:
Next:
- We need to turn Location, Fuel_Type, Transmission, Owner_Type into categorical columns.
- We need to remove units from Mileage, Engine, and Power columns and convert to them float column.
In [17]:
# removing the units from Mileage, Engine, and Power and changing the coluumn datatype to float
df['Power'] = df.Power.str.split().str.get(0).astype('float64')
df['Engine'] = df.Engine.str.split().str.get(0).astype('float64')
df['Mileage'] = df.Mileage.str.split().str.get(0).astype('float64')
# taking a look at the changes we've made to confirm it worked correctly
df.head()
Out[17]:
In [18]:
# converting Location, Fuel_Type, Transmission, and Owner_Type to categorical variables with one-hot encoding
df2 = pd.get_dummies(df, columns=['Location'])
df3 = pd.get_dummies(df2, columns=['Fuel_Type'])
df4 = pd.get_dummies(df3, columns=['Transmission'])
df5 = pd.get_dummies(df4, columns=['Owner_Type'])
#checking the changes made
df5.head()
Out[18]:
In [19]:
# converting Car_Brand and Car_Model to categorical variables with one-hot encoding
df6 = pd.get_dummies(df5, columns=['Car_Brand'])
df7 = pd.get_dummies(df6, columns=['Car_Model'])
Now we will do a Univariate & Bivariate analysis and treat Missing Values & Outliers¶
We will start by looking at the individual columns. (Univariate Analysis)
In [20]:
# installing pandas profiling library
!pip install -U pandas_profiling
In [21]:
# importing pandas profiler
from pandas_profiling import ProfileReport
In [22]:
# generating a pandas profile report to visualize the data / get a further understanding
df.profile_report()
Out[22]:
Obersvations about Columns:
- Location mode is Mumbai. Top 5 most frequent values are Mumbai (count: 949), Hyderbad (count: 876), Coimbatore (count: 772), Kochi (count: 772), Pune (count: 765)
- Year value ranges from 1996-2019, 2 modes; first at at 2015 and second at 2011. Year has a high correlation with Car_Age (understandably) and Car_Brand, so we will need to drop two of the columnns before modeling.
- Kilometers_Driven ranges from 171 to 6,500,000, mean is 58699.06. We see that Kilometers_Driven is highly skewed (possibly due to the presence of outliers which we will check for) and has a high correlatiion with Car_Age so we will need to drop one of the columns before modelng.
- Fuel_Type most to least frequent values: Diesel (count: 3852), Petrol (count: 3325), CNG (count: 62), LPG (count: 12), Electric (count: 2). Fuel type has a high correlation with Mileage andd Engine so we will need to remove two of the columns before modeling.
- Transmission only has two distinct values: Manual (count: 5204) and Automatic (count: 2049). Transmission has a high correlation with Engine, Power, New_Price, Price, Car_Brand so some of the columns will need to be dropped before modeling.
- Owner_Type most to least frequent values: First (count: 5952), Second (count: 1152), Third (count: 137), Fourt & Above (count: 12).
- Mileage ranges from 0 to 33.54. There are 2 missing values in the columnn which will need to be treated and the Mileage column has a high correlation with Fuel_Type, Engine, Power, Price, Car_Brand so some of the columns will need to be dropped before modeling. Mileage has a mean of 18.14 and a median of 18.16 so the distribution is pretty normal.
- Engine values range from 72 to 5998, with a mean of 1616.57 and median of 1493 so it is noticeably right-skewed. There are 46 missing values in the column which we will need to treat. The Engine column has a high correlation with Fuel_Type, Transmission, Mileage, Power, Seats, New_Price, Price, Car_Brand so some of the columns will need to be dropped before modeling.
- Power values range from 34.2 to 616, with a mean of 112.765 and median of 94 so it is noticeably right-skewed. We see there are 175 missing values in the column we will need to treat. The Power column has a high correlation with Transmission, Mileage, Engine, New_Price, Price, Car_Brand so some of the columns will needd to be dropped before modeling.
- Seats (ranging from 0-10) most frequent to least frequent values are: 5 seats (count: 6047), 7 seats (count: 796), 8 seats (count: 170), 4 seats (count: 119), 6 seats (count: 38), 2 seats (count: 18), 10 seats (count: 8), 9 seats (count: 3), 0 seats (count: 1). The Seats column has a high correlation with Engine and Car_Brand so two of the columns will need to be dropped before modeling. There are 53 missing values in the Seats column so they wll need to be treated. The Seats column has a mean of ~5.27972 and a median of 5.
- New_Price ranges from 3.91 to 375, with a mean of ~21.30732 and median of 11.3 so it is noticeably right-skewed. The New_Price column has a high correlation with Transmission, Engine, Power, Price, and Car_Brand so some of the columns will need to be dropped before modeling.
- Price ranges from 0.44 to 160, with a mean of ~9.479 and a median of 5.64 so it is noticeably right-skewed. There are 1234 missing values in the dataset which will need to be treated. The Price column has a high correlation with Transmission, Mileage, Engine, Power, New_Price, and Car_Brand so some of the columns will need to be dropped before modeling.
- Car_Age ranges from 3 to 26, with a mean of 8.63 and a median of 8 so we can see it is slightly right-skewed. Car_Age has a high correlaton with the Year column (understandbly) so we will need to remove one of the columns before modeling.
- Car_Model 10 most frequent values are: Maruti Swift Dzire (count: 189), Hyundai Grand i10 (count: 179), Maruti Wagon R (count: 178), Toyota Innova 2.5 (count: 145), Hyundai Verna 1.6 (count: 127), Honda City 1.5 (count: 122), Honda City i (count: 115), Hyundai Creta 1.6 (count: 110), Mercedes-Benz New C-Class (count: 110), BMW 3 Series (count: 109). The column has high cardinality (many unique values), this makes sense because there are many different Car models, even within each brand.
- Car_Brand 10 most frequent values are: Maruti (count: 1444), Hyundai (count: 1340), Honda (count: 743), Toyota (count: 507), Mercedes-Benz (count: 380), Volkswagen (count: 374), Ford (count: 351), Mahindra (count: 331), BMW (count: 312), Audi (count: 285). Car_Brand has a high correlation with Year, Transmission, Mileage, Engine, Power, Seats, New_Price, Price so some of the columns will need to be dropped before modeling.
Illustrated Insights¶
Year Column
In [23]:
# Year Column
sns.histplot(data=df, x='Year', kde=True);
Kilometers_Driven Column
In [24]:
# Kilometers_Driven Column
sns.boxplot(data=df, x='Kilometers_Driven');
In [25]:
# Kilometers_Driven Column
sns.histplot(data=df, x='Kilometers_Driven', kde=False);
We see that the entire data is highly skewed and being affected by the presence of an outlier(s) which seems to be really far out. Let's see if the box-plot improves when we set show fliers to false.
In [26]:
# boxplot of Kilometers driven without outliers
sns.boxplot(data=df, x='Kilometers_Driven', showfliers = False);
We see this has helped significantly, it would be best to look more into the outlier(s) in this columnn so they don't negatively impact our model.
In [27]:
# Treating outlier(s) in Kilometers_Driven
# function to return lower and upper bounds of IQR
def outlier_treatment(column):
sorted(column)
Q1,Q3 = np.percentile(column , [25,75])
IQR = Q3 - Q1
lower_range = Q1 - (1.5 * IQR)
upper_range = Q3 + (1.5 * IQR)
return lower_range,upper_range
# using the function to return the lower and upper IQR bounds for Kilometers_Driven
lowerbound,upperbound = outlier_treatment(df['Kilometers_Driven'])
print(lowerbound, upperbound)
In [28]:
# returning the rows where Kilometers_Driven contains outliers
df[(df['Kilometers_Driven'] < lowerbound) | (df['Kilometers_Driven'] > upperbound)]
Out[28]:
In [29]:
# function to set outliers to the lower and upper bounds of IQR respectively
def treat_outliers(df, col):
Q1 = df[col].quantile(0.25) # lower bound of IQR
Q3 = df[col].quantile(0.75) # upper bound of IQR
IQR = Q3 - Q1 # IQR
lower_whisker = Q1 - 1.5 * IQR
upper_whisker = Q3 + 1.5 * IQR
# all the values < lower bound will be set to the value of the lower bound
# all the values > upper bound will be set to the value of the upper bound
df[col] = np.clip(df[col], lower_whisker, upper_whisker)
return df
# treating outliers using the function
df = treat_outliers(df,'Kilometers_Driven')
# visualizing the column after outlier treatment
sns.boxplot(data=df,x='Kilometers_Driven')
plt.show()
We see our box-plot looks normal without having to set fliers to false so this coluumn looks good now. Let's plot a histogram to double check our changes.
In [30]:
sns.histplot(data=df, x='Kilometers_Driven', kde=True);
We see a significant improvement from before.
Mileage Column
In [31]:
# Mileage Column
sns.histplot(data=df, x='Mileage', kde=True);
Engine Column
In [32]:
# Engine Column
sns.histplot(data=df, x='Engine', kde=True);
Power Column
In [33]:
# Power Column
sns.histplot(data=df, x='Power', kde=True);
Seats Column
In [34]:
# Seats Column
sns.histplot(data=df, x='Seats', kde=True);
New_Price Column
In [35]:
# New Price Column
sns.histplot(data=df, x='New_Price', kde=True);
Price Column
In [36]:
# Price Column
sns.histplot(data=df, x='Price', kde=True);
Car_Age Column
In [37]:
# Car_Age Column
sns.histplot(data=df, x='Car_Age', kde=True);
Car_Brand Column
In [38]:
# Car_Brand Column
sns.countplot(data=df, x='Car_Brand');
plt.xticks(rotation=90)
Out[38]:
In [39]:
# calculating the number of unique Models for each Brand
Cargroups = df.groupby('Car_Brand')['Car_Model']
Cargroups.nunique()
Cargroups2 = Cargroups.nunique()
Cargroups2
Out[39]:
Location Column
In [40]:
# Location Column
sns.countplot(data=df, x='Location');
plt.xticks(rotation=90)
Out[40]:
Fuel_Type Column
In [41]:
# Fuel_Type Column
sns.countplot(data=df, x='Fuel_Type');
plt.xticks(rotation=90)
Out[41]:
Transmission Column
In [42]:
# Transmission Column
sns.countplot(data=df, x='Transmission');
plt.xticks(rotation=90)
Out[42]:
Owner_Type Column
In [43]:
# Owner_Type Column
sns.countplot(data=df, x='Owner_Type');
plt.xticks(rotation=90)
Out[43]:
Now we will do a Bivariate analysis and observe relationships between different columns in the dataset.
In [44]:
# creating a pair plot to observe relationships between the columns
sns.pairplot(data=df)
Out[44]:
In [45]:
# creating a heatmat to observe relationships between the columns
sns.heatmap(data=df.corr(), annot=True);
The heatmap and pairplots confirm the observations made about correlations made from the Pandas Profile Report. This will help us later with feature extraction.
Relationship Between Car_Brand and Price
In [46]:
sns.scatterplot(data=df, x='Car_Brand', y='Price', hue="Location");
plt.xticks(rotation=90);
plt.legend(bbox_to_anchor=(1.5, 1), borderaxespad=0)
Out[46]:
In [47]:
sns.lineplot(data = df , x = 'Car_Brand' , y = 'Price');
plt.xticks(rotation=90);
Relationship between Price and Location
In [48]:
sns.scatterplot(data=df, x='Location', y='Price');
plt.xticks(rotation=90);
In [49]:
sns.lineplot(data = df , x = 'Location' , y = 'Price');
plt.xticks(rotation=90);
Relationship between Price and Transmission
In [50]:
sns.scatterplot(data=df, x='Transmission', y='Price');
plt.xticks(rotation=90);
In [51]:
sns.lineplot(data = df , x = 'Transmission' , y = 'Price');
plt.xticks(rotation=90);
Relationship between Price and Fuel_Type
In [52]:
sns.scatterplot(data=df, x='Fuel_Type', y='Price');
plt.xticks(rotation=90);
In [53]:
sns.lineplot(data = df , x = 'Fuel_Type' , y = 'Price');
plt.xticks(rotation=90);
Relationship between Price and Owner_Type
In [54]:
sns.scatterplot(data=df, x='Owner_Type', y='Price');
plt.xticks(rotation=90);
In [55]:
sns.lineplot(data = df , x = 'Owner_Type' , y = 'Price');
plt.xticks(rotation=90);
Treating Missing Values & Feature Extraction¶
In [56]:
# lets take another look at which columns having missing values again
df.isnull().sum().sort_values(ascending=False)
Out[56]:
- There are 5 columns in the dataset with missing values (Price; 1234, Power: 175, Seats: 53, Engine: 46, Mileage: 2).
Mileage Column
In [57]:
# extracting information of other variables where Mileage is null
df.loc[df['Mileage'].isnull()==True]
Out[57]:
We see both rows with null values for Mileage have Automatic Transmission, Electric Fuel Type, and First Owner_Type. We can impute the missing values based off this information and using the Car_Model and Car_Brand. The distribution of Mileage is normal for the most part besides some larger values at the lower bound, so we will impute with the median.
In [58]:
df.groupby(['Car_Model','Car_Brand'])[['Transmission','Owner_Type','Fuel_Type','Mileage']].median()
Out[58]:
In [59]:
# df7['Mileage'] = df7['Mileage'].fillna(value = df7.groupby(['Car_Model','Car_Brand'])['Mileage'].transform('median'))
# df = df.groupby(['Car_Model','Car_Brand'])[['Transmission','Owner_Type','Fuel_Type','Mileage']].median()
# df7['Mileage'] = df7['Mileage'].fillna(value = df7.groupby(['Car_Brand','Car_Model'])['Mileage'].transform('mean'))
# df7['Mileage'] = df7['Mileage'].fillna(df7.groupby(['Car_Model', 'Car_Brand'])['Mileage'].transform('median'))
# Did not end up using this code ^
# Kept getting errors trying to impute with GroupBy so I decided to impute values based on the mean or median of the columns
In [60]:
# The Mileage column is normally distributed so we will impute missng values with the mean
# Checking the mean of the Mileage column
df7['Mileage'].mean()
Out[60]:
In [61]:
# Imputing the missing values in Mileage with the mean
df7['Mileage'].fillna(value=df7['Mileage'].mean(), inplace=True)
Engine Column
In [62]:
# The Engine column's distribution is skewed so we will impute missing values with the median
# Checking the median of the Engine column
df7['Engine'].median()
Out[62]:
In [63]:
# Imputing the missing values in Engine with the median
df7['Engine'].fillna(value=df7['Engine'].median(), inplace=True)
Seats Column
In [64]:
# The Seats column is normally distributed so we will impute missing values with the mean
# Checking the mean of the Seats column
df7['Seats'].mean()
Out[64]:
In [65]:
# Imputing the missing values in Seats with the mean
df7['Seats'].fillna(value=df7['Seats'].mean(), inplace=True)
Power Column
In [66]:
# The Power column's distribution is skewed so we will impute missing values with the median
# Checking the median of the Engine column
df7['Power'].median()
Out[66]:
In [67]:
# Imputing the missing values in Power with the median
df7['Power'].fillna(value=df7['Power'].median(), inplace=True)
Price Column
In [68]:
# The Price column's distribution is skewed so we will impute missing values with the median
# Checking the median of the Price column
df7['Price'].median()
Out[68]:
In [69]:
# Imputing the missing values in Price with the median
df7['Price'].fillna(value=df7['Price'].median(), inplace=True)
In [70]:
# Checking that the values have been imputed correctly
df7.isnull().sum().sort_values(ascending=False)
Out[70]:
Everything looks good so we will now move onto feature extraction.
In [71]:
# first we will drop the Year column as it is highly correlated with Car_Age
df7.drop(['Year'], axis=1, inplace=True)
In [72]:
# next we will drop the Power column as it is highly correlated with Engine, New_Price, Price
df7.drop(['Power'], axis=1, inplace=True)
Building the Model
In [73]:
# dropping Price from X to prepare for modelling
X = df7.drop("Price", axis=1)
y = df7["Price"]
In [74]:
# splitting the dataset into training set (70%) and test set (30%)
X_train, X_test, y_train, y_test = train_test_split( X, y, test_size=0.30, random_state=1)
In [75]:
# fitting the model to the training set
regression_model = LinearRegression()
regression_model.fit(X_train, y_train)
Out[75]:
In [76]:
# calculating the R-squared value of the training set
print(regression_model.score(X_train, y_train),)
In [77]:
# calculating the RMSE value of the training set
print(np.sqrt(mean_squared_error(y_train, regression_model.predict(X_train))))
In [78]:
# calculating the R-squared value of the test set
print(regression_model.score(X_test, y_test),)
In [79]:
# calculating the RMSE value of thee test set
print(np.sqrt(mean_squared_error(y_test, regression_model.predict(X_test))))
In [80]:
# taking a look at the coefficients and intercepts
equationinfo = pd.DataFrame(
np.append(regression_model.coef_, regression_model.intercept_),
index=X_train.columns.tolist() + ["Intercept"],
columns=["Coefficients"],
)
equationinfo
Out[80]:
In [81]:
# exporting the linear regression equation
Equation = "Price = " + str(regression_model.intercept_)
print(Equation, end=" ")
for i in range(len(X_train.columns)):
if i != len(X_train.columns) - 1:
print(
"+ (",
regression_model.coef_[i],
")*(",
X_train.columns[i],
")",
end=" ",
)
else:
print("+ (", regression_model.coef_[i], ")*(", X_train.columns[i], ")")
Since the Price column is skewed, we will modify the Price column to log of Price, rebuild the model, & compare the score to the original model.
In [82]:
# creating a copy of the dataframe
df8 = df7.copy(deep=True)
In [83]:
# checking the new dataframe
df8.head()
Out[83]:
In [84]:
# creating a new column for the log of the Price column
df8['PriceLog'] = np.log(df8['Price'])
In [85]:
# dropping the original price column from df8
df8.drop(['Price'], axis=1, inplace=True)
Now we will rebuild the model using the new dataframe wth the PriceLog column.
In [86]:
# dropping PriceLog from X to prepare for modelling
X2 = df8.drop("PriceLog", axis=1)
y2 = df8["PriceLog"]
In [87]:
# splitting the dataset into training set (70%) and test set (30%)
X2_train, X2_test, y2_train, y2_test = train_test_split( X2, y2, test_size=0.30, random_state=1)
In [88]:
# fitting the model to the training set
regression_model2 = LinearRegression()
regression_model2.fit(X2_train, y2_train)
Out[88]:
In [89]:
# calculating the R-squared value of the training set
print(regression_model2.score(X2_train, y2_train),)
In [90]:
# calculating the RMSE value of thee training set
print(np.sqrt(mean_squared_error(y2_train, regression_model2.predict(X2_train))))
In [91]:
# calculating the R-squared value of the test set
print(regression_model2.score(X2_test, y2_test),)
In [92]:
# calculating the RMSE value of the test set
print(np.sqrt(mean_squared_error(y2_test, regression_model2.predict(X2_test))))
We see that the R-squared values have improved significantly for both the training and test set in the model using the PriceLog column. The distance between the R-squared values of the training and test set has decreased too (~16 in the original model vs ~9 in the model using PriceLog).
Let's now look at the coefficients and intercept, and get the linear regression equation of the new model with PriceLog.
In [97]:
# taking a look at the coefficients and intercepts
equationinfo2 = pd.DataFrame(
np.append(regression_model2.coef_, regression_model2.intercept_),
index=X2_train.columns.tolist() + ["Intercept"],
columns=["Coefficients"],
)
equationinfo2
Out[97]:
In [95]:
# exporting the linear regression equation
Equation = "PriceLog = " + str(regression_model2.intercept_)
print(Equation, end=" ")
for i in range(len(X2_train.columns)):
if i != len(X2_train.columns) - 1:
print(
"+ (",
regression_model2.coef_[i],
")*(",
X2_train.columns[i],
")",
end=" ",
)
else:
print("+ (", regression_model2.coef_[i], ")*(", X2_train.columns[i], ")")
Insights and Key Takeaways¶
- The Brand which sells the most cars is Maruti followed by Hyundai, Honda, Toyota, Mercedes-Benz, Volkswagen, Ford, Mahindra, BMW, Audi
- The Car Model sold the most is Maruti Swift Dzire, followed by Hyundai Grand i10, Maruti Wagon R, Toyota Innova 2.5, Hyundai Verna 1.6, Honda City 1.5 , Honda City i, Hyundai Creta 1.6, Mercedes-Benz New C-Class, BMW 3 Series.
- The location which sells the most cars is Mumbai, followed by Hyderabad, Coimbatore, Kochi, and Pune.
- The age of the cars range from 3 to 26, with a mean of 8.63 and a median of 8.
- The manufacturing year of the used cars ranges from 1996-2019, with peaks of frequent values at 2015 and 2011.
- The number of Kilometers driven ranges from 171 to 6,500,000, and the mean value is 58699.06.
- The values for the Price of the used cars range from 0.44 to 160 INR, with a mean of ~9.479 and a median of 5.64.
- The values for the Price of a new car of the same model range from 3.91 to 375 INR, with a mean of ~21.30732 and median of 11.3.
- First-owned cars are the most common followed by Second-owned, Third-owned, then Fourth & Above
- Mileage values range from 0 to 33.54 with a median of 18.16 and mean of 18.14.
- The displacement volume of the engine in CC ranges from 72 to 5998, with a mean of 1616.57 and median of 1493.
- The most frequent value for number of seats (ranging from 0 to 10) is 5 seats followed by 7 seats, 8 seats, 4 seats, 6 seats, 2 seats, 10 seats, 9 seats, 0 seats.
- The values for the maximum Power of the engine in bhp range from 34.2 to 616, with a mean of 112.765 and median of 94.
- Diesel is the most frequent fuel type followed by Petrol, CNG, LPG, Electric.
- There are more cars with Manual transmission than automatic (almost twice as many)